Summary

The current version of MapReader contains unused/untested code for using the patch context when training the model. From the 2021 paper:

"Context-aware patchwork method.
A limitation of the patchwork method is that it ignores neighboring patches when training or performing model inference. To capture the strong, spatial dependencies between neighboring patches (which will be common in maps for many types of labels [16]), MapReader supports building model ensembles as shown in Fig. 8. Model-1 and Model-2 are two CV models with neural network architectures defined by the user. These models can have different or similar architectures. For exam- ple, in one of our experiments, Model-2 was a pre-trained model on our patch dataset while Model-1 was a pre-trained model from PyTorch Image Models [40] with a different architecture. As shown in Fig. 8, a patch and its context image are fed into Model-2 and Model-1, respectively. In practice, the user only specifies the size of the context image, and MapReader extracts and preprocesses the context image from the dataset. Model-1 and Model-2 generate vector representations for the input images, V1 and V2. The size of these vectors are defined by the user and a combination of the two (e.g., by concatenation) is then fed into Model-3 for prediction. Such model ensembles can be an efficient approach to achieving high- performing CV models [37]."

This PR updates this code to be more simple/user-friendly.

Fixes #287
Addresses #17

Describe your changes

• Both PatchDataset and PatchContextDataset return image(s) as tuples. These datasets now return (img,) and (img, context_img) respectively.
• Classifier handles "inputs" as tuples (so now can handle both context and normal inputs)
• Add context_dataset arg when creating datasets to enable creating of 'PatchContextDataset' type datasets from the annotations loader.
• Update twoParrellelModels class to clarify patch/context branches of the model
• Update generate_layerwise_lrs method for context datasets

Checklist before assigning a reviewer (update as needed)

• Self-review code
• Ensure submission passes current tests
• Add tests
• Update relevant docs

Reviewer checklist

Please add anything you want reviewers to specifically focus/comment on.

• Everything looks ok?